Skip to content

feat(stats)!: add telemetry for per-field cardinality limits - #2227

Merged
Eldolfin merged 15 commits into
oscarld/stats-per-key-cardinality-limitfrom
oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator
Jul 29, 2026
Merged

feat(stats)!: add telemetry for per-field cardinality limits#2227
Eldolfin merged 15 commits into
oscarld/stats-per-key-cardinality-limitfrom
oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator

Conversation

@Eldolfin

@Eldolfin Eldolfin commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Send telemetry about per-field cardinality collapses. But unlike #2212, now using a single count metric with all the offending fields in the bucket instead of separate count metrics.

Motivation

What inspired you to submit this pull request?

Additional Notes

Anything else we should know when reviewing?

How to test the change?

Describe here in detail how the change can be validated.

@Eldolfin
Eldolfin changed the base branch from main to oscarld/stats-per-key-cardinality-limit July 10, 2026 15:15
@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Jul 10, 2026 •
edited by datadog-prod-us1-3 Bot
Loading

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 97.25%
• Overall Coverage: 74.94% (+0.07%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 382b157 | Docs | Datadog PR Page | Give us feedback!

@github-actions

github-actions Bot commented Jul 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

📚 Documentation Check Results

⚠️ 4476 documentation warning(s) found

📦 datadog-ipc - 1003 warning(s)

📦 libdd-data-pipeline-ffi - 1343 warning(s)

📦 libdd-data-pipeline - 1232 warning(s)

📦 libdd-trace-stats - 898 warning(s)

Updated: 2026-07-29 13:30:34 UTC | Commit: db99d09 | missing-docs job results

@github-actions

github-actions Bot commented Jul 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🔒 Cargo Deny Results

⚠️ 11 issue(s) found, showing only errors (advisories, bans, sources)

📦 datadog-ipc - 3 error(s)

Show output 
error[unmaintained]: Bincode is unmaintained
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:25:1
   │
25 │ bincode 1.3.3 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unmaintained advisory detected
   │
   ├ ID: RUSTSEC-2025-0141
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2025-0141
   ├ Due to a doxxing and harassment incident, the bincode team has taken the decision to cease development permanently.
     
     The team considers version 1.3.3 a complete version of bincode that is not in need of any updates.
     
     ## Alternatives to consider
     
     * [wincode](https://crates.io/crates/wincode)
     * [postcard](https://crates.io/crates/postcard)
     * [bitcode](https://crates.io/crates/bitcode)
     * [rkyv](https://crates.io/crates/rkyv)
   ├ Announcement: https://git.sr.ht/~stygianentity/bincode/tree/v3.0/item/README.md
   ├ Solution: No safe upgrade is available!
   ├ bincode v1.3.3
     └── datadog-ipc v0.1.0

error[vulnerability]: Invalid pointer dereference in `fmt::Pointer` impl for `Atomic` and `Shared` when the underlying pointer is invalid
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:66:1
   │
66 │ crossbeam-epoch 0.9.18 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0204
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0204
   ├ Affected versions of `fmt::Display` dereference the underlying pointer. This causes a invalid pointer dereference e.g., when a pointer created with `Atomic::null` or `Shared::null`. `fmt::Debug` impls and pre-0.9 `fmt::Display` impls, which do not dereference pointers, are not affected by this issue.
   ├ Announcement: https://github.com/crossbeam-rs/crossbeam/pull/1276
   ├ Solution: Upgrade to >=0.9.20 (try `cargo update -p crossbeam-epoch`)
   ├ crossbeam-epoch v0.9.18
     └── crossbeam-deque v0.8.5
         └── rayon-core v1.12.1
             └── rayon v1.10.0
                 └── criterion v0.5.1
                     ├── (dev) datadog-ipc v0.1.0
                     ├── libdd-common v5.1.0
                     │   ├── datadog-ipc v0.1.0 (*)
                     │   ├── libdd-capabilities-impl v3.0.0
                     │   │   ├── libdd-shared-runtime v2.0.0
                     │   │   │   └── libdd-trace-stats v6.0.0
                     │   │   │       └── datadog-ipc v0.1.0 (*)
                     │   │   ├── libdd-trace-stats v6.0.0 (*)
                     │   │   └── libdd-trace-utils v9.0.0
                     │   │       ├── libdd-trace-obfuscation v5.0.0
                     │   │       │   └── libdd-trace-stats v6.0.0 (*)
                     │   │       ├── libdd-trace-stats v6.0.0 (*)
                     │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
                     │   ├── libdd-shared-runtime v2.0.0 (*)
                     │   ├── libdd-trace-obfuscation v5.0.0 (*)
                     │   ├── libdd-trace-stats v6.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-ddsketch v1.1.0
                     │   ├── datadog-ipc v0.1.0 (*)
                     │   └── libdd-trace-stats v6.0.0 (*)
                     ├── (dev) libdd-trace-normalization v3.0.0
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-trace-obfuscation v5.0.0 (*)
                     ├── (dev) libdd-trace-stats v6.0.0 (*)
                     └── (dev) libdd-trace-utils v9.0.0 (*)

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:205:1
    │
205 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by [@lopopolo](https://github.com/lopopolo)) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── (dev) libdd-common v5.1.0
      │   ├── datadog-ipc v0.1.0
      │   ├── libdd-capabilities-impl v3.0.0
      │   │   ├── libdd-shared-runtime v2.0.0
      │   │   │   └── libdd-trace-stats v6.0.0
      │   │   │       └── datadog-ipc v0.1.0 (*)
      │   │   ├── libdd-trace-stats v6.0.0 (*)
      │   │   └── libdd-trace-utils v9.0.0
      │   │       ├── libdd-trace-obfuscation v5.0.0
      │   │       │   └── libdd-trace-stats v6.0.0 (*)
      │   │       ├── libdd-trace-stats v6.0.0 (*)
      │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
      │   ├── libdd-shared-runtime v2.0.0 (*)
      │   ├── libdd-trace-obfuscation v5.0.0 (*)
      │   ├── libdd-trace-stats v6.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-ddsketch v1.1.0
      │   ├── datadog-ipc v0.1.0 (*)
      │   └── libdd-trace-stats v6.0.0 (*)
      ├── (dev) libdd-trace-normalization v3.0.0
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-trace-stats v6.0.0 (*)
      ├── libdd-trace-utils v9.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-tinybytes v1.1.1
              ├── datadog-ipc v0.1.0 (*)
              ├── (dev) libdd-tinybytes v1.1.1 (*)
              └── libdd-trace-utils v9.0.0 (*)

advisories FAILED, bans ok, sources ok

📦 libdd-data-pipeline-ffi - 3 error(s)

Show output 
error[vulnerability]: Invalid pointer dereference in `fmt::Pointer` impl for `Atomic` and `Shared` when the underlying pointer is invalid
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:78:1
   │
78 │ crossbeam-epoch 0.9.18 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0204
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0204
   ├ Affected versions of `fmt::Display` dereference the underlying pointer. This causes a invalid pointer dereference e.g., when a pointer created with `Atomic::null` or `Shared::null`. `fmt::Debug` impls and pre-0.9 `fmt::Display` impls, which do not dereference pointers, are not affected by this issue.
   ├ Announcement: https://github.com/crossbeam-rs/crossbeam/pull/1276
   ├ Solution: Upgrade to >=0.9.20 (try `cargo update -p crossbeam-epoch`)
   ├ crossbeam-epoch v0.9.18
     └── crossbeam-deque v0.8.5
         └── rayon-core v1.12.1
             └── rayon v1.10.0
                 └── criterion v0.5.1
                     ├── libdd-common v5.1.0
                     │   ├── libdd-capabilities-impl v3.0.0
                     │   │   ├── libdd-data-pipeline v7.0.0
                     │   │   │   └── libdd-data-pipeline-ffi v38.0.0
                     │   │   ├── libdd-data-pipeline-ffi v38.0.0 (*)
                     │   │   ├── libdd-shared-runtime v2.0.0
                     │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   ├── libdd-data-pipeline-ffi v38.0.0 (*)
                     │   │   │   ├── libdd-dogstatsd-client v4.0.0
                     │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   │   └── libdd-trace-stats v6.0.0
                     │   │   │   │       └── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   ├── libdd-telemetry v6.0.0
                     │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
                     │   │   ├── libdd-trace-stats v6.0.0 (*)
                     │   │   └── libdd-trace-utils v9.0.0
                     │   │       ├── libdd-data-pipeline v7.0.0 (*)
                     │   │       ├── (dev) libdd-data-pipeline-ffi v38.0.0 (*)
                     │   │       ├── libdd-trace-obfuscation v5.0.0
                     │   │       │   └── libdd-trace-stats v6.0.0 (*)
                     │   │       ├── libdd-trace-stats v6.0.0 (*)
                     │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
                     │   ├── libdd-common-ffi v38.0.0
                     │   │   └── libdd-data-pipeline-ffi v38.0.0 (*)
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   ├── libdd-dogstatsd-client v4.0.0 (*)
                     │   ├── libdd-shared-runtime v2.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   ├── libdd-trace-obfuscation v5.0.0 (*)
                     │   ├── libdd-trace-stats v6.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-data-pipeline v7.0.0 (*)
                     ├── (dev) libdd-ddsketch v1.1.0
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   └── libdd-trace-stats v6.0.0 (*)
                     ├── (dev) libdd-trace-normalization v3.0.0
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-trace-obfuscation v5.0.0 (*)
                     ├── (dev) libdd-trace-stats v6.0.0 (*)
                     └── (dev) libdd-trace-utils v9.0.0 (*)

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:226:1
    │
226 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by [@lopopolo](https://github.com/lopopolo)) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── libdd-common v5.1.0
      │   ├── libdd-capabilities-impl v3.0.0
      │   │   ├── libdd-data-pipeline v7.0.0
      │   │   │   └── libdd-data-pipeline-ffi v38.0.0
      │   │   ├── libdd-data-pipeline-ffi v38.0.0 (*)
      │   │   ├── libdd-shared-runtime v2.0.0
      │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   ├── libdd-data-pipeline-ffi v38.0.0 (*)
      │   │   │   ├── libdd-dogstatsd-client v4.0.0
      │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   │   └── libdd-trace-stats v6.0.0
      │   │   │   │       └── libdd-data-pipeline v7.0.0 (*)
      │   │   │   ├── libdd-telemetry v6.0.0
      │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   │   └── libdd-trace-stats v6.0.0 (*)
      │   │   │   └── libdd-trace-stats v6.0.0 (*)
      │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
      │   │   ├── libdd-trace-stats v6.0.0 (*)
      │   │   └── libdd-trace-utils v9.0.0
      │   │       ├── libdd-data-pipeline v7.0.0 (*)
      │   │       ├── (dev) libdd-data-pipeline-ffi v38.0.0 (*)
      │   │       ├── libdd-trace-obfuscation v5.0.0
      │   │       │   └── libdd-trace-stats v6.0.0 (*)
      │   │       ├── libdd-trace-stats v6.0.0 (*)
      │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
      │   ├── libdd-common-ffi v38.0.0
      │   │   └── libdd-data-pipeline-ffi v38.0.0 (*)
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   ├── libdd-dogstatsd-client v4.0.0 (*)
      │   ├── libdd-shared-runtime v2.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   ├── libdd-trace-obfuscation v5.0.0 (*)
      │   ├── libdd-trace-stats v6.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-data-pipeline v7.0.0 (*)
      ├── (dev) libdd-ddsketch v1.1.0
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   └── libdd-trace-stats v6.0.0 (*)
      ├── (dev) libdd-trace-normalization v3.0.0
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-trace-stats v6.0.0 (*)
      ├── libdd-trace-utils v9.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-tinybytes v1.1.1
              ├── libdd-data-pipeline v7.0.0 (*)
              ├── libdd-data-pipeline-ffi v38.0.0 (*)
              ├── (dev) libdd-tinybytes v1.1.1 (*)
              └── libdd-trace-utils v9.0.0 (*)

error[vulnerability]: Denial of Service via Stack Exhaustion
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:296:1
    │
296 │ time 0.3.41 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0009
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0009
    ├ ## Impact
      
      When user-provided input is provided to any type that parses with the RFC 2822 format, a denial of
      service attack via stack exhaustion is possible. The attack relies on formally deprecated and
      rarely-used features that are part of the RFC 2822 format used in a malicious manner. Ordinary,
      non-malicious input will never encounter this scenario.
      
      ## Patches
      
      A limit to the depth of recursion was added in v0.3.47. From this version, an error will be returned
      rather than exhausting the stack.
      
      ## Workarounds
      
      Limiting the length of user input is the simplest way to avoid stack exhaustion, as the amount of
      the stack consumed would be at most a factor of the length of the input.
    ├ Announcement: https://github.com/time-rs/time/blob/main/CHANGELOG.md#0347-2026-02-05
    ├ Solution: Upgrade to >=0.3.47 (try `cargo update -p time`)
    ├ time v0.3.41
      └── tracing-appender v0.2.3
          └── libdd-log v1.0.0
              └── (dev) libdd-data-pipeline v7.0.0
                  └── libdd-data-pipeline-ffi v38.0.0

advisories FAILED, bans ok, sources ok

📦 libdd-data-pipeline - 3 error(s)

Show output 
error[vulnerability]: Invalid pointer dereference in `fmt::Pointer` impl for `Atomic` and `Shared` when the underlying pointer is invalid
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:75:1
   │
75 │ crossbeam-epoch 0.9.18 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0204
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0204
   ├ Affected versions of `fmt::Display` dereference the underlying pointer. This causes a invalid pointer dereference e.g., when a pointer created with `Atomic::null` or `Shared::null`. `fmt::Debug` impls and pre-0.9 `fmt::Display` impls, which do not dereference pointers, are not affected by this issue.
   ├ Announcement: https://github.com/crossbeam-rs/crossbeam/pull/1276
   ├ Solution: Upgrade to >=0.9.20 (try `cargo update -p crossbeam-epoch`)
   ├ crossbeam-epoch v0.9.18
     └── crossbeam-deque v0.8.5
         └── rayon-core v1.12.1
             └── rayon v1.10.0
                 └── criterion v0.5.1
                     ├── libdd-common v5.1.0
                     │   ├── libdd-capabilities-impl v3.0.0
                     │   │   ├── libdd-data-pipeline v7.0.0
                     │   │   ├── libdd-shared-runtime v2.0.0
                     │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   ├── libdd-dogstatsd-client v4.0.0
                     │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   │   └── libdd-trace-stats v6.0.0
                     │   │   │   │       └── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   ├── libdd-telemetry v6.0.0
                     │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
                     │   │   ├── libdd-trace-stats v6.0.0 (*)
                     │   │   └── libdd-trace-utils v9.0.0
                     │   │       ├── libdd-data-pipeline v7.0.0 (*)
                     │   │       ├── libdd-trace-obfuscation v5.0.0
                     │   │       │   └── libdd-trace-stats v6.0.0 (*)
                     │   │       ├── libdd-trace-stats v6.0.0 (*)
                     │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   ├── libdd-dogstatsd-client v4.0.0 (*)
                     │   ├── libdd-shared-runtime v2.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   ├── libdd-trace-obfuscation v5.0.0 (*)
                     │   ├── libdd-trace-stats v6.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-data-pipeline v7.0.0 (*)
                     ├── (dev) libdd-ddsketch v1.1.0
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   └── libdd-trace-stats v6.0.0 (*)
                     ├── (dev) libdd-trace-normalization v3.0.0
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-trace-obfuscation v5.0.0 (*)
                     ├── (dev) libdd-trace-stats v6.0.0 (*)
                     └── (dev) libdd-trace-utils v9.0.0 (*)

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:218:1
    │
218 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by [@lopopolo](https://github.com/lopopolo)) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── libdd-common v5.1.0
      │   ├── libdd-capabilities-impl v3.0.0
      │   │   ├── libdd-data-pipeline v7.0.0
      │   │   ├── libdd-shared-runtime v2.0.0
      │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   ├── libdd-dogstatsd-client v4.0.0
      │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   │   └── libdd-trace-stats v6.0.0
      │   │   │   │       └── libdd-data-pipeline v7.0.0 (*)
      │   │   │   ├── libdd-telemetry v6.0.0
      │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   │   └── libdd-trace-stats v6.0.0 (*)
      │   │   │   └── libdd-trace-stats v6.0.0 (*)
      │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
      │   │   ├── libdd-trace-stats v6.0.0 (*)
      │   │   └── libdd-trace-utils v9.0.0
      │   │       ├── libdd-data-pipeline v7.0.0 (*)
      │   │       ├── libdd-trace-obfuscation v5.0.0
      │   │       │   └── libdd-trace-stats v6.0.0 (*)
      │   │       ├── libdd-trace-stats v6.0.0 (*)
      │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   ├── libdd-dogstatsd-client v4.0.0 (*)
      │   ├── libdd-shared-runtime v2.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   ├── libdd-trace-obfuscation v5.0.0 (*)
      │   ├── libdd-trace-stats v6.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-data-pipeline v7.0.0 (*)
      ├── (dev) libdd-ddsketch v1.1.0
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   └── libdd-trace-stats v6.0.0 (*)
      ├── (dev) libdd-trace-normalization v3.0.0
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-trace-stats v6.0.0 (*)
      ├── libdd-trace-utils v9.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-tinybytes v1.1.1
              ├── libdd-data-pipeline v7.0.0 (*)
              ├── (dev) libdd-tinybytes v1.1.1 (*)
              └── libdd-trace-utils v9.0.0 (*)

error[vulnerability]: Denial of Service via Stack Exhaustion
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:287:1
    │
287 │ time 0.3.41 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0009
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0009
    ├ ## Impact
      
      When user-provided input is provided to any type that parses with the RFC 2822 format, a denial of
      service attack via stack exhaustion is possible. The attack relies on formally deprecated and
      rarely-used features that are part of the RFC 2822 format used in a malicious manner. Ordinary,
      non-malicious input will never encounter this scenario.
      
      ## Patches
      
      A limit to the depth of recursion was added in v0.3.47. From this version, an error will be returned
      rather than exhausting the stack.
      
      ## Workarounds
      
      Limiting the length of user input is the simplest way to avoid stack exhaustion, as the amount of
      the stack consumed would be at most a factor of the length of the input.
    ├ Announcement: https://github.com/time-rs/time/blob/main/CHANGELOG.md#0347-2026-02-05
    ├ Solution: Upgrade to >=0.3.47 (try `cargo update -p time`)
    ├ time v0.3.41
      └── tracing-appender v0.2.3
          └── libdd-log v1.0.0
              └── (dev) libdd-data-pipeline v7.0.0

advisories FAILED, bans ok, sources ok

📦 libdd-trace-stats - 2 error(s)

Show output 
error[vulnerability]: Invalid pointer dereference in `fmt::Pointer` impl for `Atomic` and `Shared` when the underlying pointer is invalid
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:66:1
   │
66 │ crossbeam-epoch 0.9.18 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0204
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0204
   ├ Affected versions of `fmt::Display` dereference the underlying pointer. This causes a invalid pointer dereference e.g., when a pointer created with `Atomic::null` or `Shared::null`. `fmt::Debug` impls and pre-0.9 `fmt::Display` impls, which do not dereference pointers, are not affected by this issue.
   ├ Announcement: https://github.com/crossbeam-rs/crossbeam/pull/1276
   ├ Solution: Upgrade to >=0.9.20 (try `cargo update -p crossbeam-epoch`)
   ├ crossbeam-epoch v0.9.18
     └── crossbeam-deque v0.8.5
         └── rayon-core v1.12.1
             └── rayon v1.10.0
                 └── criterion v0.5.1
                     ├── libdd-common v5.1.0
                     │   ├── libdd-capabilities-impl v3.0.0
                     │   │   ├── libdd-shared-runtime v2.0.0
                     │   │   │   ├── libdd-telemetry v6.0.0
                     │   │   │   │   └── libdd-trace-stats v6.0.0
                     │   │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
                     │   │   ├── libdd-trace-stats v6.0.0 (*)
                     │   │   └── libdd-trace-utils v9.0.0
                     │   │       ├── libdd-trace-obfuscation v5.0.0
                     │   │       │   └── libdd-trace-stats v6.0.0 (*)
                     │   │       ├── libdd-trace-stats v6.0.0 (*)
                     │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
                     │   ├── libdd-dogstatsd-client v4.0.0
                     │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   ├── libdd-shared-runtime v2.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   ├── libdd-trace-obfuscation v5.0.0 (*)
                     │   ├── libdd-trace-stats v6.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-ddsketch v1.1.0
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   └── libdd-trace-stats v6.0.0 (*)
                     ├── (dev) libdd-trace-normalization v3.0.0
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-trace-obfuscation v5.0.0 (*)
                     ├── (dev) libdd-trace-stats v6.0.0 (*)
                     └── (dev) libdd-trace-utils v9.0.0 (*)

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:198:1
    │
198 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by [@lopopolo](https://github.com/lopopolo)) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── libdd-common v5.1.0
      │   ├── libdd-capabilities-impl v3.0.0
      │   │   ├── libdd-shared-runtime v2.0.0
      │   │   │   ├── libdd-telemetry v6.0.0
      │   │   │   │   └── libdd-trace-stats v6.0.0
      │   │   │   └── libdd-trace-stats v6.0.0 (*)
      │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
      │   │   ├── libdd-trace-stats v6.0.0 (*)
      │   │   └── libdd-trace-utils v9.0.0
      │   │       ├── libdd-trace-obfuscation v5.0.0
      │   │       │   └── libdd-trace-stats v6.0.0 (*)
      │   │       ├── libdd-trace-stats v6.0.0 (*)
      │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
      │   ├── libdd-dogstatsd-client v4.0.0
      │   │   └── libdd-trace-stats v6.0.0 (*)
      │   ├── libdd-shared-runtime v2.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   ├── libdd-trace-obfuscation v5.0.0 (*)
      │   ├── libdd-trace-stats v6.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-ddsketch v1.1.0
      │   ├── libdd-telemetry v6.0.0 (*)
      │   └── libdd-trace-stats v6.0.0 (*)
      ├── (dev) libdd-trace-normalization v3.0.0
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-trace-stats v6.0.0 (*)
      ├── libdd-trace-utils v9.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-tinybytes v1.1.1
              ├── (dev) libdd-tinybytes v1.1.1 (*)
              └── libdd-trace-utils v9.0.0 (*)

advisories FAILED, bans ok, sources ok

Updated: 2026-07-29 13:31:50 UTC | Commit: db99d09 | dependency-check job results

@Eldolfin Eldolfin mentioned this pull request Jul 21, 2026
Closed
@Eldolfin
Eldolfin force-pushed the oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator branch from 456c9bf to 295db59 Compare July 21, 2026 12:01
@pr-commenter

pr-commenter Bot commented Jul 21, 2026 •
edited
Loading

Copy link
Copy Markdown

Benchmarks

Comparison

Benchmark execution time: 2026-07-29 13:42:00

Comparing candidate commit 382b157 in PR branch oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator with baseline commit e3b0de2 in branch oscarld/stats-per-key-cardinality-limit.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 24 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Benchmark execution time: 2026-07-29 13:38:03

Comparing candidate commit 382b157 in PR branch oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator with baseline commit e3b0de2 in branch oscarld/stats-per-key-cardinality-limit.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz 382b157 1785331683 oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
trace_buffer/1_senders/10us_delay execution_time 59.923ms 60.080ms ± 0.036ms 60.069ms ± 0.013ms 60.098ms 60.142ms 60.204ms 60.240ms 0.28% 1.331 5.286 0.06% 0.003ms 1 200
trace_buffer/1_senders/10us_delay throughput 14940.256op/s 14979.952op/s ± 9.012op/s 14982.723op/s ± 3.215op/s 14984.513op/s 14989.173op/s 14990.173op/s 15019.272op/s 0.24% -1.321 5.281 0.06% 0.637op/s 1 200
trace_buffer/1_senders/1us_delay execution_time 50.225ms 50.650ms ± 0.254ms 50.586ms ± 0.143ms 50.781ms 51.203ms 51.340ms 51.374ms 1.56% 0.949 0.398 0.50% 0.018ms 1 200
trace_buffer/1_senders/1us_delay throughput 17518.542op/s 17769.387op/s ± 88.533op/s 17791.579op/s ± 50.559op/s 17834.898op/s 17879.275op/s 17911.398op/s 17919.447op/s 0.72% -0.927 0.352 0.50% 6.260op/s 1 200
trace_buffer/1_senders/no_delay execution_time 354.309µs 363.185µs ± 5.975µs 362.681µs ± 3.596µs 366.431µs 369.636µs 375.145µs 419.657µs 15.71% 4.371 38.195 1.64% 0.422µs 1 200
trace_buffer/1_senders/no_delay throughput 2144607.497op/s 2478702.559op/s ± 38327.725op/s 2481521.464op/s ± 24640.762op/s 2505085.501op/s 2523438.325op/s 2530092.830op/s 2540154.872op/s 2.36% -3.458 27.231 1.54% 2710.179op/s 1 200
trace_buffer/2_senders/10us_delay execution_time 60.070ms 60.126ms ± 0.040ms 60.119ms ± 0.022ms 60.140ms 60.223ms 60.264ms 60.286ms 0.28% 1.683 3.316 0.07% 0.003ms 1 200
trace_buffer/2_senders/10us_delay throughput 29857.705op/s 29937.217op/s ± 19.912op/s 29940.638op/s ± 10.999op/s 29951.925op/s 29958.478op/s 29961.876op/s 29965.010op/s 0.08% -1.678 3.297 0.07% 1.408op/s 1 200
trace_buffer/2_senders/1us_delay execution_time 50.958ms 51.204ms ± 0.078ms 51.200ms ± 0.053ms 51.257ms 51.329ms 51.408ms 51.483ms 0.55% 0.227 1.000 0.15% 0.006ms 1 200
trace_buffer/2_senders/1us_delay throughput 34963.299op/s 35153.814op/s ± 53.566op/s 35156.212op/s ± 36.361op/s 35189.344op/s 35236.404op/s 35279.999op/s 35323.295op/s 0.48% -0.213 0.990 0.15% 3.788op/s 1 200
trace_buffer/2_senders/no_delay execution_time 1.495ms 1.613ms ± 0.044ms 1.610ms ± 0.034ms 1.647ms 1.681ms 1.701ms 1.754ms 8.95% 0.134 -0.263 2.71% 0.003ms 1 200
trace_buffer/2_senders/no_delay throughput 1026107.651op/s 1117068.128op/s ± 30334.584op/s 1117930.150op/s ± 23457.050op/s 1138777.610op/s 1165096.240op/s 1183063.038op/s 1204182.601op/s 7.72% 0.004 -0.319 2.71% 2144.979op/s 1 200
trace_buffer/4_senders/10us_delay execution_time 60.129ms 60.184ms ± 0.037ms 60.178ms ± 0.020ms 60.197ms 60.259ms 60.329ms 60.367ms 0.31% 1.879 5.154 0.06% 0.003ms 1 200
trace_buffer/4_senders/10us_delay throughput 59635.644op/s 59816.686op/s ± 36.504op/s 59822.755op/s ± 19.424op/s 59840.716op/s 59858.114op/s 59863.179op/s 59871.077op/s 0.08% -1.872 5.120 0.06% 2.581op/s 1 200
trace_buffer/4_senders/1us_delay execution_time 51.155ms 51.349ms ± 0.103ms 51.341ms ± 0.060ms 51.398ms 51.527ms 51.702ms 51.824ms 0.94% 1.301 3.548 0.20% 0.007ms 1 200
trace_buffer/4_senders/1us_delay throughput 69465.757op/s 70108.696op/s ± 140.555op/s 70119.273op/s ± 81.382op/s 70203.932op/s 70293.193op/s 70349.752op/s 70374.978op/s 0.36% -1.278 3.449 0.20% 9.939op/s 1 200
trace_buffer/4_senders/no_delay execution_time 3.741ms 3.805ms ± 0.026ms 3.804ms ± 0.018ms 3.822ms 3.848ms 3.876ms 3.900ms 2.50% 0.435 0.409 0.69% 0.002ms 1 200
trace_buffer/4_senders/no_delay throughput 923147.745op/s 946077.875op/s ± 6529.568op/s 946264.529op/s ± 4381.345op/s 950683.059op/s 956572.025op/s 958895.793op/s 962317.620op/s 1.70% -0.389 0.335 0.69% 461.710op/s 1 200
trace_buffer/8_senders/10us_delay execution_time 60.252ms 60.319ms ± 0.053ms 60.307ms ± 0.024ms 60.332ms 60.427ms 60.487ms 60.579ms 0.45% 1.735 3.928 0.09% 0.004ms 1 200
trace_buffer/8_senders/10us_delay throughput 118853.178op/s 119365.311op/s ± 104.500op/s 119389.045op/s ± 48.157op/s 119436.702op/s 119485.090op/s 119493.442op/s 119497.668op/s 0.09% -1.728 3.889 0.09% 7.389op/s 1 200
trace_buffer/8_senders/1us_delay execution_time 51.548ms 51.932ms ± 0.233ms 51.863ms ± 0.137ms 52.080ms 52.383ms 52.503ms 52.555ms 1.34% 0.780 -0.287 0.45% 0.016ms 1 200
trace_buffer/8_senders/1us_delay throughput 136998.145op/s 138645.116op/s ± 620.501op/s 138827.847op/s ± 366.867op/s 139110.938op/s 139400.510op/s 139528.324op/s 139675.477op/s 0.61% -0.765 -0.312 0.45% 43.876op/s 1 200
trace_buffer/8_senders/no_delay execution_time 7.167ms 7.308ms ± 0.055ms 7.302ms ± 0.034ms 7.343ms 7.407ms 7.448ms 7.484ms 2.48% 0.358 0.354 0.75% 0.004ms 1 200
trace_buffer/8_senders/no_delay throughput 962113.743op/s 985277.010op/s ± 7437.046op/s 985974.743op/s ± 4674.106op/s 990367.228op/s 996332.615op/s 1002978.437op/s 1004549.026op/s 1.88% -0.308 0.317 0.75% 525.879op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
trace_buffer/1_senders/10us_delay execution_time [60.075ms; 60.085ms] or [-0.008%; +0.008%] None None None
trace_buffer/1_senders/10us_delay throughput [14978.703op/s; 14981.201op/s] or [-0.008%; +0.008%] None None None
trace_buffer/1_senders/1us_delay execution_time [50.615ms; 50.685ms] or [-0.069%; +0.069%] None None None
trace_buffer/1_senders/1us_delay throughput [17757.117op/s; 17781.657op/s] or [-0.069%; +0.069%] None None None
trace_buffer/1_senders/no_delay execution_time [362.357µs; 364.013µs] or [-0.228%; +0.228%] None None None
trace_buffer/1_senders/no_delay throughput [2473390.705op/s; 2484014.414op/s] or [-0.214%; +0.214%] None None None
trace_buffer/2_senders/10us_delay execution_time [60.120ms; 60.131ms] or [-0.009%; +0.009%] None None None
trace_buffer/2_senders/10us_delay throughput [29934.458op/s; 29939.977op/s] or [-0.009%; +0.009%] None None None
trace_buffer/2_senders/1us_delay execution_time [51.193ms; 51.214ms] or [-0.021%; +0.021%] None None None
trace_buffer/2_senders/1us_delay throughput [35146.390op/s; 35161.237op/s] or [-0.021%; +0.021%] None None None
trace_buffer/2_senders/no_delay execution_time [1.606ms; 1.619ms] or [-0.377%; +0.377%] None None None
trace_buffer/2_senders/no_delay throughput [1112864.046op/s; 1121272.209op/s] or [-0.376%; +0.376%] None None None
trace_buffer/4_senders/10us_delay execution_time [60.179ms; 60.189ms] or [-0.008%; +0.008%] None None None
trace_buffer/4_senders/10us_delay throughput [59811.627op/s; 59821.745op/s] or [-0.008%; +0.008%] None None None
trace_buffer/4_senders/1us_delay execution_time [51.335ms; 51.363ms] or [-0.028%; +0.028%] None None None
trace_buffer/4_senders/1us_delay throughput [70089.216op/s; 70128.175op/s] or [-0.028%; +0.028%] None None None
trace_buffer/4_senders/no_delay execution_time [3.802ms; 3.809ms] or [-0.096%; +0.096%] None None None
trace_buffer/4_senders/no_delay throughput [945172.939op/s; 946982.810op/s] or [-0.096%; +0.096%] None None None
trace_buffer/8_senders/10us_delay execution_time [60.312ms; 60.326ms] or [-0.012%; +0.012%] None None None
trace_buffer/8_senders/10us_delay throughput [119350.828op/s; 119379.793op/s] or [-0.012%; +0.012%] None None None
trace_buffer/8_senders/1us_delay execution_time [51.900ms; 51.965ms] or [-0.062%; +0.062%] None None None
trace_buffer/8_senders/1us_delay throughput [138559.121op/s; 138731.112op/s] or [-0.062%; +0.062%] None None None
trace_buffer/8_senders/no_delay execution_time [7.300ms; 7.316ms] or [-0.105%; +0.105%] None None None
trace_buffer/8_senders/no_delay throughput [984246.307op/s; 986307.713op/s] or [-0.105%; +0.105%] None None None

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 382b157 1785331683 oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 9.588ms 9.617ms ± 0.018ms 9.613ms ± 0.010ms 9.627ms 9.652ms 9.665ms 9.690ms 0.80% 1.011 1.080 0.18% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [9.615ms; 9.620ms] or [-0.025%; +0.025%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 382b157 1785331683 oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.062µs 1.078µs ± 0.007µs 1.079µs ± 0.004µs 1.082µs 1.087µs 1.094µs 1.100µs 1.99% -0.010 0.403 0.61% 0.000µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [1.077µs; 1.079µs] or [-0.085%; +0.085%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 382b157 1785331683 oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 15.471µs 15.861µs ± 0.204µs 15.841µs ± 0.111µs 15.947µs 16.186µs 16.568µs 17.002µs 7.33% 1.619 5.568 1.28% 0.014µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [15.832µs; 15.889µs] or [-0.178%; +0.178%] None None None

Baseline

Baseline benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz e3b0de2 1785331635 oscarld/stats-per-key-cardinality-limit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
trace_buffer/1_senders/10us_delay execution_time 60.017ms 60.090ms ± 0.050ms 60.074ms ± 0.010ms 60.088ms 60.202ms 60.269ms 60.379ms 0.51% 2.523 7.794 0.08% 0.004ms 1 200
trace_buffer/1_senders/10us_delay throughput 14905.848op/s 14977.422op/s ± 12.457op/s 14981.603op/s ± 2.587op/s 14983.426op/s 14987.823op/s 14989.121op/s 14995.851op/s 0.10% -2.514 7.728 0.08% 0.881op/s 1 200
trace_buffer/1_senders/1us_delay execution_time 50.093ms 50.446ms ± 0.186ms 50.401ms ± 0.097ms 50.522ms 50.724ms 51.160ms 51.213ms 1.61% 1.577 3.818 0.37% 0.013ms 1 200
trace_buffer/1_senders/1us_delay throughput 17573.677op/s 17841.236op/s ± 65.279op/s 17856.662op/s ± 34.497op/s 17882.033op/s 17921.179op/s 17944.382op/s 17966.699op/s 0.62% -1.541 3.666 0.36% 4.616op/s 1 200
trace_buffer/1_senders/no_delay execution_time 356.778µs 365.264µs ± 5.872µs 363.900µs ± 2.079µs 366.936µs 372.080µs 379.175µs 427.821µs 17.57% 6.408 63.086 1.60% 0.415µs 1 200
trace_buffer/1_senders/no_delay throughput 2103681.291op/s 2464549.720op/s ± 36214.975op/s 2473206.212op/s ± 14111.846op/s 2484536.652op/s 2496123.097op/s 2509897.343op/s 2522577.447op/s 2.00% -5.362 48.132 1.47% 2560.785op/s 1 200
trace_buffer/2_senders/10us_delay execution_time 60.076ms 60.134ms ± 0.047ms 60.122ms ± 0.026ms 60.151ms 60.235ms 60.271ms 60.299ms 0.30% 1.230 0.897 0.08% 0.003ms 1 200
trace_buffer/2_senders/10us_delay throughput 29851.016op/s 29933.282op/s ± 23.458op/s 29939.188op/s ± 12.725op/s 29950.787op/s 29958.069op/s 29960.422op/s 29962.220op/s 0.08% -1.227 0.887 0.08% 1.659op/s 1 200
trace_buffer/2_senders/1us_delay execution_time 50.718ms 51.147ms ± 0.106ms 51.153ms ± 0.062ms 51.214ms 51.306ms 51.405ms 51.454ms 0.59% -0.336 1.329 0.21% 0.007ms 1 200
trace_buffer/2_senders/1us_delay throughput 34982.950op/s 35193.030op/s ± 72.891op/s 35188.829op/s ± 42.493op/s 35231.300op/s 35308.336op/s 35369.337op/s 35490.401op/s 0.86% 0.356 1.361 0.21% 5.154op/s 1 200
trace_buffer/2_senders/no_delay execution_time 1.588ms 1.671ms ± 0.038ms 1.670ms ± 0.026ms 1.696ms 1.738ms 1.767ms 1.770ms 6.00% 0.200 -0.321 2.30% 0.003ms 1 200
trace_buffer/2_senders/no_delay throughput 1017062.455op/s 1077461.363op/s ± 24733.949op/s 1078092.942op/s ± 16885.650op/s 1095151.993op/s 1118068.696op/s 1131528.259op/s 1133627.031op/s 5.15% -0.088 -0.381 2.29% 1748.954op/s 1 200
trace_buffer/4_senders/10us_delay execution_time 60.130ms 60.191ms ± 0.039ms 60.183ms ± 0.021ms 60.203ms 60.276ms 60.307ms 60.345ms 0.27% 1.286 1.668 0.07% 0.003ms 1 200
trace_buffer/4_senders/10us_delay throughput 59656.866op/s 59809.999op/s ± 38.955op/s 59817.976op/s ± 20.797op/s 59838.409op/s 59854.168op/s 59862.261op/s 59870.757op/s 0.09% -1.282 1.655 0.06% 2.755op/s 1 200
trace_buffer/4_senders/1us_delay execution_time 51.192ms 51.368ms ± 0.117ms 51.346ms ± 0.052ms 51.405ms 51.608ms 51.825ms 51.880ms 1.04% 1.728 4.001 0.23% 0.008ms 1 200
trace_buffer/4_senders/1us_delay throughput 69391.006op/s 70082.230op/s ± 159.528op/s 70112.116op/s ± 70.701op/s 70180.455op/s 70259.923op/s 70319.869op/s 70324.164op/s 0.30% -1.708 3.907 0.23% 11.280op/s 1 200
trace_buffer/4_senders/no_delay execution_time 3.794ms 3.859ms ± 0.025ms 3.861ms ± 0.017ms 3.876ms 3.903ms 3.919ms 3.931ms 1.81% 0.125 -0.037 0.64% 0.002ms 1 200
trace_buffer/4_senders/no_delay throughput 915842.519op/s 932832.203op/s ± 6020.385op/s 932464.667op/s ± 4175.251op/s 937326.096op/s 942113.792op/s 946358.793op/s 948970.183op/s 1.77% -0.087 -0.052 0.64% 425.705op/s 1 200
trace_buffer/8_senders/10us_delay execution_time 60.249ms 60.323ms ± 0.075ms 60.300ms ± 0.027ms 60.339ms 60.440ms 60.489ms 60.996ms 1.16% 4.249 31.917 0.12% 0.005ms 1 200
trace_buffer/8_senders/10us_delay throughput 118039.894op/s 119356.791op/s ± 146.871op/s 119403.725op/s ± 54.130op/s 119446.161op/s 119476.443op/s 119487.680op/s 119504.881op/s 0.08% -4.191 31.174 0.12% 10.385op/s 1 200
trace_buffer/8_senders/1us_delay execution_time 51.422ms 51.783ms ± 0.210ms 51.731ms ± 0.120ms 51.897ms 52.210ms 52.310ms 52.419ms 1.33% 0.870 0.152 0.40% 0.015ms 1 200
trace_buffer/8_senders/1us_delay throughput 137355.934op/s 139043.212op/s ± 561.999op/s 139181.480op/s ± 324.397op/s 139427.056op/s 139763.147op/s 139910.467op/s 140018.327op/s 0.60% -0.853 0.119 0.40% 39.739op/s 1 200
trace_buffer/8_senders/no_delay execution_time 7.129ms 7.242ms ± 0.059ms 7.239ms ± 0.030ms 7.268ms 7.302ms 7.351ms 7.834ms 8.21% 4.973 48.171 0.81% 0.004ms 1 200
trace_buffer/8_senders/no_delay throughput 919126.838op/s 994232.307op/s ± 7823.195op/s 994611.708op/s ± 4050.404op/s 998694.415op/s 1004353.534op/s 1006346.639op/s 1009954.410op/s 1.54% -4.385 40.777 0.78% 553.183op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
trace_buffer/1_senders/10us_delay execution_time [60.084ms; 60.097ms] or [-0.012%; +0.012%] None None None
trace_buffer/1_senders/10us_delay throughput [14975.696op/s; 14979.148op/s] or [-0.012%; +0.012%] None None None
trace_buffer/1_senders/1us_delay execution_time [50.420ms; 50.471ms] or [-0.051%; +0.051%] None None None
trace_buffer/1_senders/1us_delay throughput [17832.188op/s; 17850.283op/s] or [-0.051%; +0.051%] None None None
trace_buffer/1_senders/no_delay execution_time [364.450µs; 366.078µs] or [-0.223%; +0.223%] None None None
trace_buffer/1_senders/no_delay throughput [2459530.673op/s; 2469568.767op/s] or [-0.204%; +0.204%] None None None
trace_buffer/2_senders/10us_delay execution_time [60.127ms; 60.140ms] or [-0.011%; +0.011%] None None None
trace_buffer/2_senders/10us_delay throughput [29930.031op/s; 29936.533op/s] or [-0.011%; +0.011%] None None None
trace_buffer/2_senders/1us_delay execution_time [51.132ms; 51.161ms] or [-0.029%; +0.029%] None None None
trace_buffer/2_senders/1us_delay throughput [35182.928op/s; 35203.132op/s] or [-0.029%; +0.029%] None None None
trace_buffer/2_senders/no_delay execution_time [1.666ms; 1.677ms] or [-0.319%; +0.319%] None None None
trace_buffer/2_senders/no_delay throughput [1074033.475op/s; 1080889.250op/s] or [-0.318%; +0.318%] None None None
trace_buffer/4_senders/10us_delay execution_time [60.185ms; 60.196ms] or [-0.009%; +0.009%] None None None
trace_buffer/4_senders/10us_delay throughput [59804.600op/s; 59815.398op/s] or [-0.009%; +0.009%] None None None
trace_buffer/4_senders/1us_delay execution_time [51.352ms; 51.385ms] or [-0.032%; +0.032%] None None None
trace_buffer/4_senders/1us_delay throughput [70060.121op/s; 70104.339op/s] or [-0.032%; +0.032%] None None None
trace_buffer/4_senders/no_delay execution_time [3.856ms; 3.863ms] or [-0.090%; +0.090%] None None None
trace_buffer/4_senders/no_delay throughput [931997.835op/s; 933666.570op/s] or [-0.089%; +0.089%] None None None
trace_buffer/8_senders/10us_delay execution_time [60.313ms; 60.334ms] or [-0.017%; +0.017%] None None None
trace_buffer/8_senders/10us_delay throughput [119336.436op/s; 119377.145op/s] or [-0.017%; +0.017%] None None None
trace_buffer/8_senders/1us_delay execution_time [51.754ms; 51.812ms] or [-0.056%; +0.056%] None None None
trace_buffer/8_senders/1us_delay throughput [138965.325op/s; 139121.100op/s] or [-0.056%; +0.056%] None None None
trace_buffer/8_senders/no_delay execution_time [7.234ms; 7.250ms] or [-0.113%; +0.113%] None None None
trace_buffer/8_senders/no_delay throughput [993148.087op/s; 995316.527op/s] or [-0.109%; +0.109%] None None None

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz e3b0de2 1785331635 oscarld/stats-per-key-cardinality-limit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 9.526ms 9.559ms ± 0.017ms 9.556ms ± 0.012ms 9.571ms 9.590ms 9.600ms 9.622ms 0.68% 0.605 0.079 0.18% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [9.557ms; 9.561ms] or [-0.025%; +0.025%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz e3b0de2 1785331635 oscarld/stats-per-key-cardinality-limit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.072µs 1.093µs ± 0.007µs 1.093µs ± 0.005µs 1.098µs 1.105µs 1.109µs 1.111µs 1.63% -0.041 -0.058 0.64% 0.000µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [1.093µs; 1.094µs] or [-0.089%; +0.089%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz e3b0de2 1785331635 oscarld/stats-per-key-cardinality-limit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 15.357µs 15.712µs ± 0.209µs 15.698µs ± 0.104µs 15.802µs 16.014µs 16.349µs 17.315µs 10.31% 2.769 16.863 1.33% 0.015µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [15.683µs; 15.741µs] or [-0.184%; +0.184%] None None None

@Eldolfin

Eldolfin commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@dd-octo-sts

dd-octo-sts Bot commented Jul 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 87.20 MB 87.33 MB +.15% (+135.77 KB) 🔍
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.01 MB 8.01 MB +0% (+48 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.78 MB 10.78 MB +.04% (+4.53 KB) 🔍
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 98.45 MB 98.59 MB +.14% (+142.12 KB) 🔍
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 26.06 MB 26.08 MB +.10% (+28.50 KB) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 89.18 KB 89.60 KB +.47% (+430 B) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 188.08 MB 188.23 MB +.07% (+152.00 KB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 979.11 MB 980.23 MB +.11% (+1.12 MB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 8.47 MB 8.48 MB +.15% (+13.50 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 89.18 KB 89.60 KB +.47% (+430 B) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 25.05 MB 25.08 MB +.15% (+40.00 KB) 🔍
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 49.83 MB 49.90 MB +.14% (+71.80 KB) 🔍
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 22.62 MB 22.65 MB +.12% (+28.50 KB) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 90.58 KB 91.00 KB +.46% (+434 B) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 192.55 MB 192.71 MB +.08% (+160.00 KB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 970.38 MB 971.14 MB +.07% (+778.24 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 6.54 MB 6.55 MB +.20% (+13.50 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 90.58 KB 91.00 KB +.46% (+434 B) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 26.91 MB 26.94 MB +.14% (+40.00 KB) 🔍
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 47.43 MB 47.50 MB +.14% (+71.32 KB) 🔍
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 77.70 MB 77.82 MB +.14% (+116.47 KB) 🔍
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.92 MB 8.93 MB +.13% (+12.06 KB) 🔍
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 93.26 MB 93.38 MB +.12% (+122.60 KB) 🔍
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.85 MB 10.87 MB +.13% (+15.23 KB) 🔍

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Jul 21, 2026

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: e1430135ee

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@Eldolfin Eldolfin changed the title feat(stats)!: add telemetry for per-field cardinality limits V2 feat(stats)!: add telemetry for per-field cardinality limits Jul 21, 2026
@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Jul 21, 2026 •
edited by datadog-prod-us1-5 Bot
Loading

Copy link
Copy Markdown

View session in Datadog

Bits Code status: ✅ Done

Comment @DataDog to request changes

@Eldolfin
Eldolfin force-pushed the oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator branch 2 times, most recently from bf05f1a to e648bf3 Compare July 22, 2026 11:05
@Eldolfin
Eldolfin marked this pull request as ready for review July 22, 2026 11:17
@Eldolfin
Eldolfin requested review from a team as code owners July 22, 2026 11:17
@Eldolfin

Eldolfin commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed Jul 23, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 59adf71d1f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread libdd-trace-stats/src/span_concentrator/cardinality_limit_telemetry.rs Outdated
yannham
yannham reviewed Jul 24, 2026
View reviewed changes
Comment thread libdd-trace-stats/src/span_concentrator/cardinality_limit_telemetry.rs Outdated

#[derive(Debug, Clone, Default, Copy)]
// Note: slot 0 is a counter for non_collapsed spans. It's not used for emitting telemetry
pub struct CollapsedFieldsMetrics(pub(super) [usize; COLLAPSED_FIELD_METRIC_SIZE]);

@yannham yannham Jul 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if I understand correctly, you don't count each collapsed field individually, but you count sets of collapsed fields instead. I'm curious about the reason behind it. I would naively expect that counting each field individually would be enough.

@VianneyRuhlmann VianneyRuhlmann Jul 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comes from the cardinality spec. Since we apply tags to increment of a global count we want the global count to be the number of collapsed spans.

This is easy to do if you increment the count when you collapse you just apply the correct tags to the increment. However this means adding a reference to the telemetry/statd client to the SpanConcentrator which is currently "pure computation logic" making it harder to reuse in serverless/sidecar.

This was found as a way to allow the StatsExporter to emit the metric by giving it for each increment the offending fields. However this arguably not much better than adding a telemetry/dogstatsd handle to the concentrator.

Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs Outdated
Comment thread libdd-trace-stats/src/span_concentrator/cardinality_limit_telemetry.rs Outdated
Comment thread libdd-trace-stats/src/span_concentrator/cardinality_limit_telemetry.rs Outdated
Comment thread libdd-trace-stats/src/span_concentrator/mod.rs Outdated
VianneyRuhlmann
VianneyRuhlmann reviewed Jul 27, 2026
View reviewed changes
Comment thread datadog-ipc/src/shm_stats.rs
Comment thread libdd-trace-stats/src/span_concentrator/cardinality_limit_telemetry.rs Outdated

#[derive(Debug, Clone, Default, Copy)]
// Note: slot 0 is a counter for non_collapsed spans. It's not used for emitting telemetry
pub struct CollapsedFieldsMetrics(pub(super) [usize; COLLAPSED_FIELD_METRIC_SIZE]);

@VianneyRuhlmann VianneyRuhlmann Jul 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comes from the cardinality spec. Since we apply tags to increment of a global count we want the global count to be the number of collapsed spans.

This is easy to do if you increment the count when you collapse you just apply the correct tags to the increment. However this means adding a reference to the telemetry/statd client to the SpanConcentrator which is currently "pure computation logic" making it harder to reuse in serverless/sidecar.

This was found as a way to allow the StatsExporter to emit the metric by giving it for each increment the offending fields. However this arguably not much better than adding a telemetry/dogstatsd handle to the concentrator.

@Eldolfin
Eldolfin force-pushed the oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator branch from e4264ec to 568df74 Compare July 28, 2026 14:21
@Eldolfin
Eldolfin force-pushed the oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator branch from 568df74 to 382b157 Compare July 29, 2026 13:28
@Eldolfin
Eldolfin merged commit e8ffd6e into oscarld/stats-per-key-cardinality-limit Jul 29, 2026
57 checks passed
@Eldolfin
Eldolfin deleted the oscarld/stats-per-key-cardinality-limit-telemetry-in-concentrator branch July 29, 2026 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yannham yannham yannham left review comments
@chatgpt-codex-connector chatgpt-codex-connector[bot] chatgpt-codex-connector[bot] left review comments
@VianneyRuhlmann VianneyRuhlmann VianneyRuhlmann approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Eldolfin @yannham @VianneyRuhlmann